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o ■ 

O ■ Abstract 

<n : 

^ , We use entropy rates and Schur concavity to prove that, for every integer k > 2, 

every nonzero rational number q, and every real number a, the base-A: expansions 
Ph ■ of a, q + a, and qa all have the same finite-state dimension and the same finite-state 

0^ \ strong dimension. This extends, and gives a new proof of, Wall's 1949 theorem 

stating that the sum or product of a nonzero rational number and a Borel normal 
• number is always Borel normal. 

1 Introduction 

The finite-state dimension of a sequence S over a finite alphabet S is an asymptotic 
^ ■ measure of the density of information in S as perceived by finite-state automata. This 
quantity, denoted dumps (50, is a finite-state effectivization of classical Hausdorff dimension 
[HI E2| introduced by Dai, Lathrop, Lutz, and Mayordomo |9,. A dual quantity, the finite- 
state strong dimension of S, denoted Dim FS (S'), is a finite-state effectivization of classical 
packing dimension [221 EH H2] introduced by Athreya, Hitchcock, Lutz, and Mayordomo 
[2j. (Explicit definitions of diniFs(S') and DimFs(S') appear in section El) In fact both 
dmiFst^) and DiniFs(S') are asymptotic measures of the density of finite-state information 
> ! in S, with < dim F s(5') < Dim F s(5') < 1 holding in general and dim F s(5') = DimFs(S') 
holding when S is sufficiently "regular." 

Although finite-state dimension and finite-state strong dimension were originally de- 
fined in terms of finite-state gamblers (HI I2j (following the gambling approach used in the 
first effectivizations of classical fractal dimension [213 E]), they have also been shown to 
admit equivalent definitions in terms of information- lossless finite-state compressors [3121; 
finite-state predictors in the log- loss model [121 [2] , and block-entropy rates . In each 
case, the definitions of dim F s(5') and Dim F s(5') are exactly dual, differing only that a 
limit inferior appears in one definition where a limit superior appears in the other. These 
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two finite-state dimensions are thus, like their counterparts in fractal geometry, robust 
quantities and not artifacts of a particular definition. 

The sequences S satisfying dimps(S') = 1 are precisely the (Borel) normal sequences, 
i.e., those sequences in which each nonempty string w G X* appears with limiting fre- 
quency |£|~' M, L (This fact was implicit in the work of Schnorr and Stimm and pointed 
out explicitly in 0.) The normal sequences, introduced by Borel in 1909 J3J, were exten- 
sively investigated in the twentieth century j2H EH EU EDI EH]- Intuitively, the normal 
sequences are those sequences that are random relative to finite-state automata. This 
statement may seem objectionable when one first learns that the Champernowne sequence 

0100011011000001010011100 

obtained by concatenating all binary strings in standard order, is normal (Hj, but it should 
be noted that a finite-state automaton scanning this sequence will spend nearly all its 
time in the middle of long strings that are random in the (stronger) sense of Kolmogorov 
complexity [19] and, having only finite memory, will have no way of "knowing" where such 
strings begin or end. This perspective is especially appropriate when modeling situations 
in which a data stream is truly massive relative to the computational resources of the 
entity processing it. 

An informative line of research on normal sequences concerns operations that preserve 
normality. For example, in his 1949 Ph.D. thesis under D.H. Lehmer, Wall [3UI proved 
that every subsequence that is selected from a normal sequence by taking all symbols 
at positions occurring in a given arithmetical progression is itself normal. Agafonov |JJ 
extended this by showing that every subsequence of a normal sequence that is selected 
using a regular language is itself normal; Kamae [Jj]] and Kamae and Weiss [T7] proved 
related results; and Merkle and Reimann [23 proved that a subsequence selected from 
a normal sequence using a context-free language need not be normal (in fact, can be 
constant, even if selected by a one-counter language). For another example, again in his 
thesis, Wall j3Dj (see also [TBI 03) proved that, for every integer k > 2, every nonzero 
rational number q, and every real number a that is normal base k (i.e., has a base-A; 
expansion that is a normal sequence), the sum q + a and the product qa are also normal 
base k. (It should be noted that a real number a may be normal in one base but not in 
another 0|2n]) 

This paper initiates the study of operations that preserve finite-state dimension and 
finite-state strong dimension. This study is related to, but distinct from, the study of op- 
erations that preserve normality. It is clear that every operation that preserves finite-state 
dimension must also preserve normality, but the converse does not hold. For example, a 
subsequence selected from a sequence according an arithmetical progression need not have 
the same finite-state dimension as the original sequence. This is because a sequence with 
finite-state dimension less than 1 may have its information content distributed heteroge- 
neously. Specifically, given a normal sequence S over the alphabet {0, 1}, define a sequence 
T whose n th bit is the § th bit of S if n is even and otherwise. Then the sequence S and 
the constant sequence 0°° are both selected from T according to arithmetic progressions, 
but it is easy to verify that dim FS (T) = Dim FS (T) = ^,dim FS (0 00 ) = Dim FS (0°°) = 0, 
and dim F s(S') = Dim FS (S') = 1. Hence, Wall's first above-mentioned theorem does not 
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extend to the preservation of finite-state dimension. Of course, this holds a fortiori for 
the stronger results by Agafonov, Kamae, and Weiss. 

Our main theorem states that Wall's second above-mentioned theorem, unlike the first 
one, does extend to the preservation of finite-state dimension. That is, we prove that, 
for every integer k > 2, every nonzero rational number q, and every real number a, the 
base-A; expansions of a, q + a, and qa all have the same finite-state dimension and the 
same finite-state strong dimension. 

The proof of our main theorem does not, and probably cannot, resemble Wall's uniform 
distribution argument. Instead we use Bourke, Hitchcock, and Vinodchandran's block- 
entropy rate characterizations of dim F s and Dim FS 0, coupled with the Schur concavity 
of the entropy function [23 122 Hi] , to prove that finite-state dimension and finite-state 
strong dimension are contractive functions with respect to a certain "logarithmic block 
dispersion" pseudometric that we define on the set of all infinite /c-ary sequences. (A 
function is contractive if the distance between its values at sequences S and T is no more 
than the pseudodistance between S and T.) This gives a general method for bounding 
the difference between the finite-state dimensions, and the finite-state strong dimensions, 
of two sequences. We then use this method to prove our main theorem. In particular, 
this gives a new proof of Wall's theorem on the sums and products of rational numbers 
with normal numbers. 

In summary, our main result is a fundamental theorem on finite-state dimension that is 
a quantitative extension of a classical theorem on normal numbers but requires a different, 
more powerful proof technique than the classical theorem. 



2 Preliminaries 

Throughout this paper, E = {0, 1, . . . , k — 1}, where k > 2 is an integer. All strings are 
elements of £*, and all sequences are elements of If x is a string or sequence and 
i,j are integers, x[i . .j] denotes the string consisting of the i th through j th symbols in x, 
provided that these symbols exist. We write x[i] — x[i . . i] for the i th symbol in x, noting 
that x[0] is the leftmost symbol in x. If to is a string and x is a string or sequence, we 
write wCito indicate that w — x[0 . . n — 1] for some nonnegative integer n. 
A base-k expansion of a real number a G [0, 1] is a sequence S G S°° such that 

oo 

a = ^S[n]£T (n+1) 



n=0 



A sequence S G S°° is (Borel) normal if, for every nonempty string w G S + 

lim — \{u G £ <n | uw C S}\ = |£|" H , 

n— >oo n 

i.e., if each string w appears with asymptotic frequency k~^ in S. 

If Q is a nonempty finite set, we write A(O) for the set of all (discrete) probability 
measures on Q, i.e., all functions 7r : Q — > [0, 1] satisfying 7r ('^) = 1- We write 

A n = A({l,...,n}). 
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All logarithms in this paper are base 2. The Shannon entropy of a probability measure 
ii e A(fi) is 

ff(7r) = 5>(ti01°g^. 

where log ^ = 0. 

We briefly define finite-state dimension and finite-state strong dimension. As noted in 
the introduction, several equivalent definitions of these dimensions are now known. In this 
paper, it is most convenient to use the definitions in terms of block-entropy rates, keeping 
in mind that Bourke, Hitchcock, and Vinodchandran [6] proved that these definitions are 
equivalent to earlier ones. 

For nonempty strings 10,16 S + , we write 



#n(w,x) 



\x\ 

m < - — 1 — 1 

\w\ 



x[m\w\ . . (m + l)\w\ — 1] 



for the number of block occurrences of w in x. Note that < #\j(w,x) < M. 

For each sequence S G positive integer n, and string w G S <ra , the n th block 
frequency of w in S is 

#n{w,S[0..n\w\-l]) 

7TS,n(w) = 



n 



Note that, for all S G E°° and < / < n, 



i.e., 7Tg n G A(E'), where we write 7Tg n for the restriction of 7Ts in to S z . 

For each sequence S G S°° and positive integer I, the / th normalized lower and upper 
block entropy rates of S are 

H r(S) = TT^rliminf H (irf n ) 

and 

^ = iiSt 1 T5 pjr (*S-)- 

respectively. 
Definition. Let S G 

1. The finite-state dimension of £ is 

dim FS (S') = inf Hj~(S). 

ZGZ+ 1 

2. The finite-state strong dimension of 5 is 

Dim FS (S) = inf H+iS). 

More discussion and properties of these dimensions appear in the references cited in 
the introduction, but this material is not needed to follow the technical arguments in the 
present paper. 
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3 Logarithmic Dispersion and Finite-State Dimen- 
sion 

In this section we prove a general theorem stating that the difference between two se- 
quences' finite-state dimensions (or finite-state strong dimensions) is bounded by a certain 
"pseudodistance" between the sequences. Recall that A n — A({1, . . . , n}) is the set of all 
probability measures on {1, . . . , n}. 

Definition. Let n be a positive integer. The logarithmic dispersion (briefly, the log- 
dispersion) between two probability measures tt, G A n is 

S(n, y) = logm, 

where m is the least positive integer for which there is an n x n nonnegative real matrix 
A = (dij) with the following three properties. 

(i) A is stochastic: each column of A sums to 1, i.e., Y^i=i a «i = 1 holds for all 1 < j < n. 

(ii) An = fi, i.e., YTj=i ^j^ti) = MO holds for all 1 < i < n. 

(iii) No row or column of A contains more than m nonzero entries. 

It is clear that 6 : A n x A n — > [0,log72,]. We now extend 5 to a normalized function 
5 + : S°° x S°° — > [0, 1]. Recall the block-frequency functions defined in section 121 

Definition. The normalized upper logarithmic block dispersion between two sequences 
S, T e S°° is 

5 + (S, T) = lim sup lim sup 5 (ir^ n!p ) . 

l^oo /logfc n^oo V ' J 

Recall that a pseudometric on a set X is a function d : X x X — > R satisfying the 
following three conditions for all x,y, z G X. 

(i) d(x, y) > 0, with equality if x — y. (nonnegativity) 

(ii) d(x,y) = d(y, x). (symmetry) 

(iii) d(x, z) < d(x, y) + d(y, z). (triangle inequality) 

(A pseudometric is a metric, or distance function, on X if it satisfies (0) with "if" replaced 
by "if and only if".) The following fact must be known, but we do not know a reference 
at the time of this writing. 

Lemma 3.1. For each positive integer n, the log- dispersion function 5 is a pseudometric 
on A n . 
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Proof. Let n be a positive integer and let tt, fx, v 6 A n . Since <5(7r, /j) = logm, where m is 
a positive integer, 5(tt, fx) > 0. Thus 5 is nonnegative. If 7r = fx, then it is easy to verify 
that the n x n identity matrix /„ testifies that 5(tt, fx) = 0. 

To show that 5 is symmetric, it suffices to prove that S(tt, fx) < 5 (fx, tt). Let m = 2 5 ^' w \ 
Then there exists an n x n nonnegative stochastic matrix A = (a^) such that tt = A fx 
and A has at most m nonzero entries in each row and column. Define the n x n matrix 
A' = (a^) for all 1 < i,j < n by 



if vr(i) > 



For all 1 < j < n such that 7r(j) = 0, 

n n j 

i=l i=l ^*=1 

Since v4/i = 7r, for all 1 < j < n such that 7r(j) > 0, 

n n \ U 1 

so A' is stochastic. Since A is stochastic, for all 1 < % < n, 

n n / / '\ \ n 

i=i i=i ^ ^vJJ/ i=1 

so = /i. Since a^j = => a\j = 0, and A has at most m nonzero entries in each 
row and column, A' has at most m nonzero entries in each row and column as well. Thus 



<5(7r, yu) < logm = 8(/j,,7t), so 5 is symmetric. 

To see that 5 satisfies the triangle inequality, let m\ = 

2«(w,m) an d m2 = 2*M. It 

suffices to show that 5(tt, v) < log m\ + log m2 = log mim.2. There exist nxn nonnegative 
stochastic matrices Ai and A 2 having no more than m\ and m 2 nonzero entries in each 
row and column, respectively, satisfying A\tt = fx and A 2 fx = v. Let A = A 2 A\. Since the 
product of two stochastic matrices is stochastic, A is stochastic. Also, An = A 2 (Aitt) = 
A 2 fx = v. Finally, since no row or column of A\ (resp. A 2 ) contains more than m\ (resp. 
m 2 ) nonzero entries, no row or column of A contains more than m\m 2 nonzero entries. 
Thus S(tt, v) < log mim2, so 5 satisfies the triangle inequality. □ 

It is easy to see that S is not a metric on A n for any 77 > 2. For example, if tt is any 
nonuniform probability measure on {1, ... ,77} and fx obtained from tt by permuting the 
values of tt nontrivially, then tt 7^ fx but 5(tt, fx) = 0. 

Lemma f3.ll has the following immediate consequence. 

Corollary 3.2. The normalized upper log-block dispersion function S + is a pseudometric 
on 
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If d is a pseudometric on a set X, then a function / : X — > K is d-contractive if, for 
all x,y & X, 

\f(x)-f(y)\<d{x,y), 

i.e., the distance between /(x) and /(y) does not exceed the pseudodistance between x 
and y. We prove the following lemma at the end of this section. 

Lemma 3.3. For each positive integer n, the Shannon entropy function 
H : A n — > [0,logn] is 5 -contractive. 

The following useful fact follows easily from Lemma 13. HI 

Theorem 3.4. Finite-state dimension and finite- state strong dimension are 5 + -contractive. 
That is, for all S,T 6E°°, 

|dim FS OS)-dim FS (T)| <5 + (S,T) 

and 

|Dim FS (S) -Dim FS (T)| < 5+{S,T). 
In this paper, we only use the following special case of Theorem 13.41 
Corollary 3.5. Let S,T G // 

limsup<n4%4>) = °(0 

as I — > oo, then 

dim FS (S') = dim FS (T) 

and 

Dim FS (S') = Dim FS (T). 

The proof of Lemma I3~3l uses Schur concavity [27J 1221 E| > which we now review. We say 
that a vector He = (xi, . . . , x n ) G M. n is nonincreasing if x\ > . . . > x n . If x , y el™ are 
nonincreasing, then we say that x majorizes y , and we write ~x )p if, if the following 
two conditions hold. 

(i) Et=i ^ = E1U ?/t- 

(ii) For all 1 < t < n, Yh=i x * ^ E*=i 2/t- 

Given a vector x G M n and a permutation 7r of {1, . . . , n}, write 7r( x ) = (x^i), . . . , x^n))- 
Call a set D C M n symmetric if 7r( x ) G D holds for every 1c E D and every permutation 
7r of {1, . . . , n}. For Z) C R n , a function / : D — > R is then symmetric if .D is symmetric 
and /( x ) = /(7r( x )) holds for every ~x E D and every permutation 7r of {1, . . . , n}. 

Definition. Let DCI" and / : £) — > K be symmetric. Then / is Schur-concave if, for 
all i,V e M n , 
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The set A n of all probability measures on {1, . . . , n} can be regarded as the (n — 1) 
dimensional simplex 



A n = { y g [0, l] n 



n 



5> = 1 ^ 



i=l 



This set A n is symmetric, as is the Shannon entropy function H : A n — > [0, logn]. In fact, 
the following fundamental property of Shannon entropy is well known [3]. 

Lemma 3.6. The Shannon entropy function H : A n — > [0,logn] Schur- concave. 

We now use Lemma f3. 61 to prove Lemma [3.31 

Proof of Lemma Vi.Sl Fix a positive integer n, and let p , q G A n . By the symmetry of 
S (established in Lemma [3 it suffices to prove that 

Htf)<Htf) + 8{?,-q)- (3-1) 

Without loss of generality, assume that p and ~q are nonincreasing. Let m be the positive 
integer such that 5(p , g ) = logm, and let A = (ay) be an n x n matrix testifying to the 
value of S(p , q). Define an n x n matrix B = (by) by 



hi 



1 if (i — l)m < j < min{zm, n}; 
otherwise. 



That is, the first block of m entries in the first row of B are l's, the second block of m 
entries in the second row of B are l's, and so on, until the last n — m (| — 1 — l) entries 

in the [—1 th row of B are l's. 

I m I 

Let r = B p . Intuitively, B represents the "worst-case" matrix with no more than 
m nonzero entries in each row and column, in the sense that it produces the vector with 
the lowest entropy. More formally, we show that majorizes the vector q , and thus r 
has entropy at most that of q . However, since B is limited to m nonzero entries in each 
row and column, it cannot redistribute the values in p by too much, so the entropy of 
~r will be close to that of p . 

Since B is stochastic (because each column contains exactly one 1) and p G A n , we 
have G A n . Clearly, is nonincreasing. For each 1 < j < n, let Cj = {j \ a,ij > 0}, 
noting that |C»| <m. Then, for all 1 < t < n, 

t t n t min{im,n} min{tm,n} 

= EE hjPj E E Vj E Pi 

i=l i=lj=l i=l j=(i— i)m+l i=l 

t t n t 

> E ft > EE fliiPi = EE a-ijPj = T.Qi- 

jeCiU...UCt i=ljeCi i=lj=l i=l 

(The first inequality holds because p is nonincreasing and each |Cj| < m. The second 
inequality holds because EILi a «i = ^ holds for each 1 < j < n, whence a single pj's 
appearances in various Cj's collectively contribute at most pj to the sum on the right.) 
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This shows that ~r )p q , whence Lemma f3. 61 tells us that H(r) < H(~q). It follows by 
Jensen's inequality and the (ordinary) concavity of the logarithm that 

HCp) < H(~p) + \og^2pi— r^i - J^log ( — J 

i=i m i=i m ' 

n n / 1 \ 

i=l i=l v ^ 7 

< F(^) + log m - Pi log Q-r j 

- 1 

= pi log h log m 

n 1 

= > rj log — h log m 
= H(r) + 6(p,-q) 

i.e., dSU) holds. □ 



4 Finite-State Dimension and Real Arithmetic 

Our main theorem concerns real numbers rather than sequences, so the following notation 
is convenient. For each real number a and each integer k > 2, write 

dinipg(a) = diniFs(S') 

and 

Dining (a) = Dim FS (6'), 

where S is a base-& expansion of a — \_a\ . Note that this notation is well-defined, because 
a real number a has two base-A; expansions if and only if it is a A;-adic rational, in which 
case both expansions are eventually periodic and hence have finite-state strong dimension 
0. It is routine to verify the following. 

Observation 4.1. For every integer k > 2, every positive integer m, and every real 
number a, 

dimpg(m + a) = dining (— a) = dining (a) 

and 

Dimp fc g(m + a) = Dining (— a) = Dining (a). 
The following lemma contains most of the technical content of our main theorem. 
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Lemma 4.2 (main lemma). For every integer k > 2, every positive integer m, and 
every real number a > 0, 



and 



dinip fc g(ma) = dinipg(a) 



Dining (ma) = Dinip fc g(a) 



Proof. Let k, m, and a be as given, let S,T e S°° be the base-k expansions of a — [_a\ . 
ma — \ma\ , respectively, and write 



H a,n H S,n ' n ma,n n T,n 



for each l,n£ Z + . By Corollary 13.51 it suffices to show that 

Iimsiiptf(7rW B , 7 r^ B )= o (0 (4.1) 



as I — > oo. 

Let r = |_l°gfc m J > let 



m = 

i=0 

be the base-fc expansion of m, and let 



i=0 

The first thing to note is that, in base k, ma — \ ma\ is the sum, modulo 1, of s copies of 
a — with of these copies shifted % symbols to the left, for each < % < r. 
For each I e Z + and j G N, let 

uf = S\jl.. (j + 1)1 — 1], 



= T[j7..(j + 1)/-1] 



be the j Z-symbol blocks of a — [a\ , ma — [ma\ , respectively. If we let 

r oo 

T f) = Y^m t S[t + i]k-^ 

i=0 t=(j+l)l 

be the sum of the tails of the above-mentioned s copies of a — [_a\ lying to the right of 
the j th /-symbol block, then the block of ma — \_ma\ is completely determined by Uj, 
the "carry" 

and the longest string of symbols shifted from the right, which is the string itj+JO . . r — 1]. 
To be more explicit, note that 

r oo 
i=0 t=(j+l)l 
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define the "advice" 



hf = [cf,uf[Q..r-\^ G {0,...,s} x £ r ; 



and define the function 

:S i x{0,...,s}xE r ^ 
by letting f^ l \x, c, z) be the base-A; expansion of the integer 

r i—1 

> (fc) +c+Y, m iYl z ^ kt mod 

i=0 t=0 



where raj^ is the nonnegative integer of which x is a base-A; expansion, possibly with 
leading O's. (Intuitively, the three terms here are the "block product", the "carry", and 
the "shift", respectively.) Then, for all integers I > and j > 0, 

vy = p'{u)',h) J ). 

For positive integers I and n, define the k l x k l matrix A^ l,n ^ = (a y l £^ by 



a 



(i,n) 



{ 



J < n 



= x and 



/« (x,hf\j))=y} 



mr^n(x) 



if 4n(^) > 

if TT^ n {x) = and x = y 
otherwise 



for all x,y e S ! . It is routine to verify that 

£ <"* = 1 

for all x G i.e., A* 7 '™) is stochastic, and that 

^1 a y,a; ^ 7r a,n(' r ) = 7r ma,n(l') 

lies' 

for all y G i.e., A^^n^n = iima,n- We complete the proof by bounding the number of 
nonzero entries in each row and column of A^ l,n \ 

Fix a column x of A^ l,n \ If Tr$n(x) = 0, then there is exactly one nonzero entry in 
column x of A <yl,n \ If iT® n {x) > 0, then the number of nonzero entries in column x is 
bounded by 

|{0, . . . , s} x S r | = (s + l)A; r < (s + l)m. 
Hence there are at most (s + l)m nonzero entries in column x of A^ l,n \ 
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Fix a row y of A^ l ' n \ Let g be the greatest common divisor of m and k l . Note that, 
for all rii, ri2 G , 



mni = mri2 mod fc z 



k l 



9 
k l 

9 



in 



(n 2 - fix) 



9 

n 2 - ni 



rii = n 2 mod 



This implies that each string y e E z has at most g preimages x under the mapping that 
takes x to the base-A; expansion of mnp mod k l . This, in turn, implies that there are at 
most g\{0, . . . , s} x S r | < g(s + l)m nonzero entries in row y of A^ l ' n \ 
We have shown that, for each /, n £ Z + , the matrix A^ 1 '^ testifies that 

* Ki^U < logG/0* + l)m) < log(m 2 ( S + 1)). 
Since this bound does not depend on / or n, this proves (j4.1j) . 



□ 

We now prove that addition and multiplication by nonzero rationals preserve finite- 
state dimension and finite-state strong dimension. 

Theorem 4.3 (main theorem). For every integer k > 2, every nonzero rational number 
q, and every real number a, 



dining (q + a) = dim F g (qa) = dining 



(*)/ 



a 



and 



Dimp fc g (q + a) = Dinipg (qa) = Dim F g j (o;). 



(*), 



.(*)/ 



Proof. Let k, q, and a be as given, and write q = | , where a and b are integers with a^O 
and b > 0. By Observation 14. II and Lemma f4. 21 



dining (qa) 



and 



dim F fc g (q + a) 



dim F fc g ( — n 
dim^g '(I a | a) 



dining 1 ^ + a 



dining ( h—<\ 
dim{§(a), 



dim 



(fc) 

FS 



dim 



(*) 

FS 



a + ba 



a + ba 



dim F g^(a + ba) 



= dim F fc g (ba) 
Similarly, Diixipg ((/a) = Dim F fc g(a), and Dimj^(g + a) 



dinipg (a) 



Dim F fc g (a). 



□ 
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Finally, we note that Theorem 14 . 31 gives a new proof of the following classical theorem. 

Corollary 4.4. (Wall \E3j ) Let k > 2. For every nonzero rational number q and every 
real number a that is normal base k, the sum q + a and the product qa are also normal 
base k. 

Acknowledgment. The authors thank Philippe Moser and Arindam Chatterjee for use- 
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